Chapter-2 Recurrence of Markov Chains - Certain Conditions for Ergodicity and Application of the Results to Queueing Problems

Let {X, n ^! 01 be an irreducible time-homogeneous MC with transition probability matrix P = (P1 ). The quantity E(X +1 X X = i ) is of interest because it gives intuitively appealing and easily computed conditions for positive recurrence. The condition lim sup E(X+ 1 X X = i) <0 was given by Pakes and the main result i-* of this paper is sharpening of this criterion for periodic MC with transition probabilities satisfying limp.. 0 for eachj. This is facilitated by considering a MC for which the matrix P can be written as the product of a finite number of stochastic matrices and obtaining an inequality satisfied by jim sup E(X +1 Xn I X = i). i—*co This inequality can also be used directly to get conditions for positive recurrence of such a MC and this is illustrated by considering a single server queueing problem with rotating servers. Let {X, n ^! 01 be a MC with stationary transition probabilities P's where P = P{X+ 1 = j I X = if and the set of non-negative integers as state space. Foster's (1953) theorem and its extension (which was first stated by Kingman (1961) and Pakes (1969)) give necessary and sufficient conditions for positive recurrence of {X, n ^! 01 when its state space forms a single irreducible class.